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O | Abstract 

We show that if DTIME[2°W] is not included in DSPACE[2°W], then, for every 

Qh! set B in PSPACE/poly, all strings x in B of length n can be represented by a string 

compressed{x) of length at most log(|B = "|) + 0(\ogn), such that a polynomial-time 

£f~) ■ algorithm, given compressed (x), can distinguish x from all the other strings in B =n . 

Modulo the 0(log«) additive term, this achieves the information-theoretic optimum 

r\ ' for string compression. We also observe that optimal compression is not possible for 

sets more complex than PSPACE/poly because for any time-constructible superpoly- 

nomial function t, there is a set A computable in space t(n) such that at least one string 

O | x of length n requires compressed(x) to be of length 2 log ( \A =n \ ) . 

^h \ Keywords: compression, time-bounded Kolmogorov complexity, pseudo-random gen- 

^ erator. 

o: 

1 Introduction 

^r ■ 

o- 

In many practical and theoretical applications in computer science, it is important to rep- 
resent information in a compressed way. If an application handles strings x from a fi- 
nite set B, it is desirable to represent every x by another shorter string compressed(x) 
^ such that compressed(x) describes unambigously the initial x. Minimizing the length of 

compressed (x) is one of the main goals of this task and ideally one would like to achieve 
the information-theoretic bound \compressed(x)\ < log(|B|), for all x E B. If a set B is 
computably enumerable, a fundamental result in Kolmogorov complexity states that for all 
x E B =n , C(x) < log(|5 = "|) + 0(logn), where C(x) is the Kolmogorov complexity of x, 
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i.e., the shortest effective description of x (B =n is the set of strings of length n in B). The 
result holds because x can be described by its rank in the enumeration of B =n . However, in 
many applications, it is desirable that the unambiguous description is not merely effective, 
but also efficient. This leads to the general idea of considering time-bounded versions of 
Kolmogorov complexity. An interesting line of research [Sip83, BFL01, BLvM05, LR05J 



in time-bounded Kolmogorov complexity, which we also pursue in this paper, focuses on a 
notion called time-bounded distinguishing Kolmogorov complexity, CD l (-), introduced by 
Sipser [Sip83|. We say that a description (called & program in the Kolmogorov complexity 
literature) p distinguishes a string x if p accepts x and only x. CD l ' A (x) is the size of the 
smallest program that distinguishes x and that runs in time t{\x\) with access to the oracle A. 
Sipser showed that, for every set B and every length n, there is a string w of length poly(n) 
such that, for every xeB =n ,CV poly ' B=n (x | w) < log(|B= n |)+loglog(|B =n |) + 0(1). Hence 
the optimal upper bound of log(|i? = "|) can be achieved (almost) if we allow the distin- 
guishing program to use an advice of polynomial length. Later, Buhrman, Fortnow, and 
Laplante [BFL01 1 showed how to avoid the advice at the expense of the length of com- 
pression. More precisely, they showed that for some polynomial p, for every set B, and 
every string x G B =n , CD pB (x) < 21og(|B =n |) + 0(logn). Hence a significant question 
is whether the factor 2 in the length of compression is necessary if no advice is given. In- 
terestingly, Buhrman, Laplante, and Miltersen [B LM001 showed that in the general setting, 
there are sets for which the answer is yes. In particular, they showed that there is a set B 
such that for sufficiently large n, there exists x G BT n for which CD tA > 21og |B =n | — 0(1) 
for a t that is super-polynomial. 

There are results in the literature where the upper bound is log(|5 = "|) (+ a small "pre- 
cision" term) at the price of weakening other parameters. For example, Buhrman, Fortnow, 
and Laplante [B FL01H showed such an upper bound of log(|5 = "|) in an average-case set- 
ting: For any B, any e, for all except a fraction of e strings x G B =n , CD poly ' B (x) < 
log(|B =n |) +polylog(n • 1/e). The precision term polylog(n ■ 1/e) has been improved to 
0(log(/z • 1/e)) in HBMVZ13H . In addition, Buhrman, Lee, and van Melkebe ek DBLvM051l 
showed that for all B and x G B =n , CNDP ol y' B=n (x) < log(|B= n |) + 0((v/log(|B= n |) + 
log n) log n), where CND is similar to CD except that the distinguisher program is non- 
deterministic. 

Our Contribution 

In this paper we consider the following question. Is it possible to achieve the optimal 
log \B =n \ bound for the language compression problem in the case where we have a bound 
on the complexity of the language we are compressing? Our main result is that, under 
a certain reasonable hardness assumption that is used in the area of derandomization, the 
upper bound of log(\B =n \) holds for every set B in the class PSPACE/poly. We state the 
result below. For a precise and stronger statement, see Theorem l3.ll 



Main Result. Assume that there exists / £ E that cannot be computed in space 2°^ n >. Then 
for any A in PSPACE/poly, there exists a polynomial p such that for every x £ A =n , 

CDP< A=n (x) <log(|A= n |) + 0(logn). 



The main result is a corollary of the following stronger result: Under the same hardness 
assumption, the distinguisher program p for x of length log(|A = "|) + 0(logn) is simple 
conditioned by x, in the sense that C poly (p \ x) = O(logn), where C poly (•) is the polynomial- 
time bounded Kolmogorov complexity function. The idea of the proof is that the hardness 
assumption implies efficient pseudo-random generators that are sufficient to derandomize 
Sipser's non-uniform result (which is a consequence of a randomized construction). 

We also show that in a natural sense the main result is optimal by showing that for any 
super-polynomial function t that is time constructible, there is a set A computable in space 
t(n) for which the lower bound from [BLMOOJ holds: For every polynomial p, for every 
sufficiently large n, there exists x £ A =n with CD p ' A ~ n (x) > 21og(|A =n |) - 0(1). 

Finally we consider the applicability of pseudo-random generator constructions for the 
problem of (efficiently) compressing/decompressing efficient languages, originally consid- 
ered by Goldberg and Sipser [GS85] and subsequently treated by Trevisan, Vadhan, and 
Zuckerman [TVZ05J. We show that the hardness assumption that there exists / £ E that 
cannot be computed in space 2°( n \ leads to improved results on this compression problem 
(see Section \5\ for precise statement of results.) 

It is natural to investigate to what extent basic results in Kolmogorov complexity remain 
valid in resource-bounded Kolmogorov complexity, especially for the case of polynomial 
bounds. In general, proofs in Kolmogorov complexity can be easily adapted for the space- 
bounded version. The case of time-bounded Kolmogorov complexity is quite different. 
Most proofs cannot be converted from the classical setting to the polynomial-time setting 
and, in fact, some results no longer hold. For example, symmetry of information does not 
hold for polynomial-time-bounded Kolmogorov complexity [LW95] (provided one-way 
permutations exist, which is generally believed to be true). 

Recently, certain techniques based on the theory of pseudo-randomness in computa- 
tional complexity have been used to obtain interesting results in polynomial-time-bounded 
Kolmogorov complexity. For instance, Antunes and Fortnow [AF09] have shown that, 
under a certain reasonable hardness assumption, 2~ CP ( X > dominates all P-samplable dis- 
tributions (where C p (x) is the length of the shortest program that generates x within time 
/?(|x|)). Our approach uses the similar machinery and reinforces the fact that this is a pow- 
erful technique in time-bounded Kolmogorov complexity research. 

2 Preliminaries 

We use \A\ to denote the size of a finite set A, [M] to denote the finite set {1, . . . ,M}, and 
log(.) to denote log 2 (.). 



We use standard notation and concepts from computational complexity and Kol- 
mogorov complexity (see [A B091 I1LV080. We are interested in languages in PSPACE/poly. 
A language A is in PSPACE/poly if there exist a machine M and a polynomial p such that 
for all input lengths n there is a string y of length p(n), and for all x of length n, x E A 
if and only if M(x,y) accepts x in p(n) space. The advice represents information that is 
nonuniform across input lengths, and is given to the machine along with the input. 

C l (x) denotes the Mime-bounded Kolmogorov complexity of x. Formally, C'(x) is the 
minimal length of a program p such that a fixed universal machine U on input p prints x in 
at most t(\x\) steps. Since a different universal machine affects the Kolmogorov complexity 
only by an 0(1) additive term, and the time bound by a 0(logt) multiplicative factor, in 
the rest of the paper we will fix one arbitrary universal machine and work with it. 

Our main focus in this paper is a variation of time-bounded Kolmogorov complexity 
called distinguishing complexity introduced by Sipser QSip83| . The Mime bounded dis- 
tinguishing complexity of x, denoted CD c (x) is the length of the shortest program p that 
accepts x and only x within time t(\x\). We define this formally next. 

Definition 2.1 (Distinguishing complexity). The t-time bounded distinguishing complexity 
ofx, denoted CD l (x), is the length of the shortest program p such that (1) U(p,x) accepts, 
(2) U(p,v) rejects for all v ^ x, and(3) U(p,v) halts in atmostt(\v\) steps for all v. 

Here U is the type of universal Turing machine typically used for time-bounded Kol- 
mogorov complexity. If U is an oracle machine, we define in a similar way CD t,A (x), by 
allowing U to query the oracle A. We fix U and we call a string p as above, a program. We 
use the notation p(x) as a substitute for U(p : x) and p A (x) as a substitute for U A {p,x) (i.e., 
A is the oracle used by program p). 

Our main tool is hardness based pseudo-random generators which we discuss next. 

Hardness assumptions and pseudo-random generators 

The proof of the main result relies on probability distributions that (1) have small support, 
(2) are efficiently samplable, and (3) cannot be distinguished from the uniform distribution 
by certain predicates. More precisely, we need pseudo-random generators that extend a 
seed of length 0(logn) to a string of length n in time polynomial in n, and such that the 
output "looks" uniformly random to certain predicates T of bounded complexity. Formally, 
a pseudo-random generator g : {0, l} clo 8" — >■ {0, 1}" fools a predicate T if 

l Prob ze{o,i} cl °g4 T (g( s )) = 1] - p r°bze{o,i}4 T ( u ) = 1]| < V n - 

By the celebrated result of Impagliazzo and Wigderson HIW971 , strengthening an ear- 
lier result of Nisan and Wigderson HNW941 , and its relativization obtained by Klivans and 
van Melkebeek [KvM02], it follows that certain hardness assumptions imply the existence 
of pseudo-random generators of the type that we need. Let / : {0, 1}* — > {0, 1} be some 
function and T C {0, 1}* be a set (viewed also as a predicate via the identification with its 
characteristic function). Let S?(«) denote the size of the smallest circuit with T gates that 



computes the function / for inputs of length n. For ^ a complexity class (such as PSPACE, 
NP, or L k , the k-th level of the polynomial hierarchy), we use Sj(n) to denote S T r{n) for 
some predicate T that is complete under polynomial-time reduction for the class *€ . We 
denote E = U c>0 DTIME[2 cn ] . 

Assumption H{T): There exists a function / in E such that, for some e > 0, SAn) > 2 En . 

Theorem 2.2 (Klivans and van Melkebeek [KvM02]). IfH{T) is true, then there exists a 
constant c and a pseudo-random generator g : {0, 1 } clo S" — > {0, 1 } n that fools the predicate 
T and such that, for every s G {0, 1 } cl °g" ) g($) is computable in time polynomial in n. 

In our main application the set T is in PSPACE/poly. For such T, one can use the 
following hardness assumption H\ that is less technical and is still plausible. 

Assumption H\ : There exists a function / in E such that, for some e > 0, S P r SPACE (n) > 2 £n . 

The following lemma is easy to prove. 

Lemma 2.3. If T e PSPACE/poly, then Hi implies H(T). 

One can also use a hardness assumption that only involves uniform computation which 
is more cleaner to state (i.e., no circuits or advice information). 

Assumption H2'. There exists a function / in E which is not computable in space 2°( n \ 

More explictly, this means that / is in E, and for every machine M that computes / 
there exists a constant £ > such that, for all sufficiently large n, M requires space at least 
2 ew , on some input of length n. 

Lemma 2.4 (Miltersen UMilOlll ). H 2 implies H\. 

3 Main result 

In this section we state and prove our main theorem. 

Theorem 3.1. Assume H\ holds. Then for every A in PSPACE/poly, there exists a polyno- 
mial p such that, for all x G A =n , CD pA " (x) < log | A =n | + O(logn). 

Proof Let A in PSPACE/poly. Fix n, and let k = [log |A =n |] . Let Jf be the set of linear 
functions h : {0, 1}" —$■ {0, 1}^ +1 . Each h G J4f is given by a. (k+l) x n matrix H over 
GF[2] and h(x) = Hx . We say that "h isolates x" if for all yeA =n \ {x}, h(x) # h(y). 
It is easy to check that, for fixed x and y in A =n , Probhe^[h(x) = h(y)] = 1/2 . Thus, 
for fixed x in A =n , Probh e ^[h does not isolate x] < |A| • l/2 k+1 < 1/2. If we take uni- 
formly at random a tuplet of (k+l) functions h = (h\,.. .,hk+i) G Jt? k+l , Probjj[no hi G 
h isolates x] < l/2 k+1 . Therefore, Prob E [(3x G A =n ) no hi G h isolates x] < |A| • l/2 k+1 < 
1/2. 

Note that given h = (hi,... ,Afc+i) such that some hi G h isolates x, x can be described 
(in the sense of CD poly ' A ()) by i and h{(x), an information which has length log£+ (^+ 1) = 
log \A =n \ + 0(logn). The problem is that the length of h is n(k+ l) 2 . 



We can obtain a shorter such h using the assumption Hi and the pseudorandom gener- 
ator implied by it. Let T(h), where h = (hi,. . . ,%-t-i) G J0" , be the predicate defined 
by 

T(h) = 1 iff (Vjc G A- n )(3ht e h,h t isolates x). (1) 

Since A is in PSPACE/poly, it is easy to check that the predicate T is computable in 
PSPACE/poly. Therefore assumption Hi implies assumption H(T), which, at its turn, im- 
plies the existence of a pseudo-random generator g : {0, l} cl °g' 7 — y {0, l}"^ 1 ) such that 

Prob s€{0)1} cio g „[T(g(s)) = 1] > Prob E [T(h) = 1] - 1/n > 1/2-1/n > 0. 

Therefore, there exists s G {0, l} clog " such that g(s) is a tuplet h = (hi,.. .,%+i) with 
the property that for every x G A, there is some i G {1, . . . ,k + 1} such that /z ; isolates x. 
Thus, any string x in A can be described by p = (k,s,i,ht(x)), an information which can 
be encoded with k + 0(logn) bits. Indeed, on input (p, v), the distinguishing algorithm 
constructs the pseudo-random generator g (which depends on k and n = \x\), calculates 
g(s) = (hi,.. .,h k+ i), and accepts if and only if /z,(v) = h{(x). By the discussion above, 
this algorithm only accepts the string x. O 

Remark. The proof shows more: The program p = (k,s,i,hj(x)) witnessing that 
CD poly ' A (x) < log |A =n | +0(logn), can be obtained from x,k,i,s in polynomial time, 
and therefore (y oly (p | x) < O(logn). 

Using the same technique, the following result for sets in the polynomial-time hierarchy 
with polynomial advice holds (the conclusion is weaker than in Theorem l3.il but so is the 
assumption). 

Theorem 3.2. Assume that there exists a function f in E such that, for some £ > 0, S } (n) > 

2 En , where k is a natural number. Then for every A in E^/poly, there exists a polynomial p 
such that, for all xeA =n , CDP' A (x) < log|A =n | +0(logn). 



4 A lower bound for sets computable in superpolynomial 
space 

We show that the complexity class PSPACE/poly is essentiallly the largest class for which 
the optimal language compression in Theorem 13. II holds. Indeed, if t(n) is a function 
that is superpolynomial and time-constructible, then there exists a set A computable in 
space t(n) for which the 2 log \A =n \ — O(l) lower bound shown by Buhrman, Laplante, and 
Miltersen HBLM00H holds. 

Theorem 4.1. Let t(n) be a superpolynomial and time-constructible function. There exists 
a set A computable in space t(n) such that for all sufficiently large n, there exists x G A =n 
such that 

(1) CD l ( n ) 1/2 ' A (x) >21og|A= n |-0(l), 



(2) \A= n \ >t{n) l / b . 

The proof follows closely the arguments from [BLMOOJ (we add only elements that 
determine the space complexity bound for the set A). For completeness, we present it 
below. The key part is a combinatorial result on fc-cover-free subsets. A fc-cover-free family 
& is a family of N subsets of [M] with the property that no subset in & is contained in the 
union of k other subsets in & . More precisely, if Fq,Fi, . . . ,F& are distinct subsets in j£~, 
then F Q <£ F\ U . . . U F k . Dyachkov and Rykov HDR82H have shown that if k < N 1 / 3 , then 

M > 2iogf+c ' f° r some constant c. 

Theorem 4.2 (Dyachkov and Rykov HDR82I0 . Let & be a family ofN subsets of[M\. Then 

if ' & is k-cover-free for k < N 1 ' 3 , then M > 21 °| , for some constant c. 

Proof of Theorem \4~1\ Let us fix n (sufficiently large) and define r = min{ ^(n) 1 / 2 ] , 2 W / 8 }, 
and k = [r 1 / 3 ] . Let & be the set of programs of length at most 21og(fc+ 1) — c\ (for a 
constant c\ that will be specified later), which run in time bounded by r. Clearly, \&\< 

First we find the lexicographically smallest string y of length r ■ n such that if y = 
x\%2 ■ ■ ■ x r , with every x; e {0, 1}", then the strings X\,...,x r are all distinct and C 3r (v) > 
r-n—l (recall that C 3r (y) is the Kolmogorov complexity of y with time bounded by 3r). By 
a simple counting argument such a string exists and, furthermore, it can be found in space 
bounded by r ■ n + 3r = o(t(n) 1 ' 2 ). Let B = {x\, . . . ,x r }. Note that for any two distinct 
string Xi,Xj G B, C(xj \ Xj) > n/2 (otherwise y could be reconstructed in 3r steps from 
information less than r ■ n — 1). 

This implies that for any set A CB, for any program p E &, and for any x E A, 

p {x} (x)=p A (x). (2) 

This is true because otherwise, the program p on input x and with oracle {x}, during its 
r-step computation, queries the oracle about some string u EA\ {x}. Note that C(u \ x) < 
logr+ \p\ +0(1) < n/2, which contradicts the property of the elements of B. 

Claim 4.3. There exists a set AC B, \A\=k+\ and xEA such that for every p E £?, either 

(1) p{**(x) does not accept, or 

(2) p{ x > (x) accepts and there exists y EA\ {x} such that pW (y) accepts. 

The statement of the theorem is a consequence of Claim 14.31 as can be seen from the 
following two observations. First, by Equation ©, for every p E £? , either 

{I) p A (x) does not accept, or 

(2) p A (x) accepts and there exists y E A \ {x} such that p A (y) accepts. 

This implies that CD r ' A (x) > 21og(k+ 1) -0(1) = 21og|A =n | -0(1). Secondly, it is 
easy to see that, by an exhaustive search, one can find a set A satisfying the conditions in 
Fact 14.31 and print out its elements in space 0(r), and thus the set A is computable in space 

t( n y/ 2 . 



Proof of Claim \4~3\ It remains to show Claim R3I To reach a contradiction, suppose that 
for every subset ACfiof size k+\ and for every x EA there exists p E & such that {u E 
A | pW (u) accepts } = {x}. We define a family of r subsets as follows: For every x E B, let 
F x = {p€ & \ p*- x *(x) accepts }. {F x \ x E B} is afamily of r subsets of {1, . . . , (k+ \) 2 /c{\ 
which by our assumption is fc-cover-free. By the combinatorial result of Diachkov and 

Rykov, ^ c ' > 2l °K C , which, for large enough c\, is a contradiction. □ 

5 Compression of efficient languages 

We show that hardness based pseudo-random generator constructions can be used to get 
improved upper bound on standard compression/decompression problem of efficient lan- 
guages first considered by Goldberg and Sipser [GS85 1 (this is different from the language 
compression problem given by the CD poly (-) complexity). In the standard compression 
problem there are two steps: (1) given a string x we seek a succinct representation of it, the 
string y (compression), and (2) given y, we want to reconstruct x (decompression). 

Definition 5.1. A set A is compressible by a uniform family of algorithms {Enc n ,Dec n ) n >\ 
to length m{n) if for each n, 

(1) Enc n : {0, 1}" -► {0, l}< n \Dec n : {0, l} m W -> {0, 1}", 

(2 ) For all x E A =n , Dec n (Enc n (x)) = x. 

Goldberg and Sipser [GS85] showed that any set A in P with \A =n \ < 2 n /n k , can be 
compressed in polynomial time to length n — (k — 3) logrc. Their algorithm is probabilis- 
tic and hence can be erroneous with some small probability. Later, Trevisan, Vadhan and 
Zuckerman [TVZ05J gave, for any set A in P, compression algorithms to length k(n) + 
polylog(n - k(n)) that run in time 2"~ fc W -poly(n) • 2P ol y 1 °g( 1 / £ ), where k(n) > [log |A= n |] 
and k(n) is computable in time poly(n). However, their algorithms only work for a (1 — e) 
fraction of strings in A =n . We show that, under the hardness assumption H\, compres- 
sion/decompression can be done for all strings in a language in P with parameters essen- 
tially identical to the above results, but without any error. 

Theorem 5.2. Assume H\. Let A be a set in P such that for every n, \A =n \ < 2 n /n 2 . Let 
k{n) > [log|A = "|] be a function computable in polynomial time on input \ n . Then A is 
compressible to length k(n) + 0(logn) by algorithms (Enc n ,Dec n ) running in time 2 n ~ k ("> ■ 
poly(n). 

Remarks: We make two remarks about this theorem. (1) If k(n) = n — 0(logn), 
(Enc n ,Dec„) run in polynomial time, are deterministic, and work for all strings in A =n . (2) 
The assumption Hi can be replaced by the (probably) weaker assumption: There exists / 
in E such that, for some e > 0, S™ > 2 En . 

Proof. The proof is a variation of the proof of Theorem 13. II and we use the notation from 
that proof. We fix length n and let k = k(n). We define the predicate t by adding to 
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the predicate T from equation (Q3, the condition that all the linear functions hi have rank 
(k+1). Thus, 

f(h) = 1) iff T(h) = 1 and (V/z ; - e h,h { has rank k + 1)]. 

The probability, when /z is chosen at random in ^* +1 , of the event "t(h) = 1" is at least 
1/3, because the probability that a random (&+ 1) x n matrix has rank less than k+ 1 is 
approximately 0(2~^ n ~ k '), and therefore the probability of the event "T(h) = 1" is only 
slightly less than the probability of the event "T(h) = 1," which we have seen to be at least 
1/2. Consequently, there exists a pseudo-random generator g^ : {0, l} cl °s n — y {0, l}"( fc + 1 )"' 
such that 

Prob se{o,i}ci°gn[t(g k (s)) = 1] > 0. 

The compression algorithm Enc n on input x E A =n works as follows: It finds a seed s such 
that if gk(s) = (h\,.. .,hk+i), there exists hi that isolates x and such that hi has rank (k+\). 
This takes time 2 n ~ k ■ poly (n) because if hi has rank (k+l), there are at most 2 n ~ k ~ l strings 
x' such that hi(x') = hj(x) . Then, the algorithm Enc n on input x returns (k, n, s, i, hj(x)). 

The decompression algorithm Dec n on input (k,n,s,i,y), constructs g k , then g k (s) = 
(hi,.. .jhk+i) and determines the set hj (y) which has at most 2 n ~ k ~ l elements. Then it 
searches for the unique element in that set that belongs to A =n , and returns that element. 

□ 
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